Enhanced Education for Better Imaging Practices: A Case Study for Improving and Understanding Existing Imaging Guidelines

Recent notable efforts to establish new technical standards and best practices for digital imaging, including the Federal Agency Digitization Guidelines Initiative in the United States and the Metamorfoze effort in the Netherlands, present important educational challenges and perhaps one of the biggest opportunities for the imaging science community to increase the level of imaging literacy in the ranks of new and upcoming cultural heritage professionals. This paper establishes the contexts for and presents the preliminary results of an educational exercise on digitization quality carried out in collaboration between academia and industry. A graduate level course introduces students to emerging standards and best practices and reinforces this information with training in the use of the GoldenThread image quality software via a server-based “virtual laboratory” environment. Recognizing that improvements in teaching imaging concepts are also needed, we present examples from an image quality interpretation manual developed to complement classroom discussion and laboratory exercises. Introduction A little Learning is a dang’rous thing; Drink deep, or taste not the Pierian Spring: There shallow Draughts intoxicate the Brain, and drinking largely sobers us again. Alexander Pope, An Essay on Criticism (1711) The lack of “imaging literacy” is perhaps the single biggest impediment to the adoption of good standardized imaging practices in the cultural heritage community. This form of applied knowledge encompasses the abilities to read, interpret, and use generally accepted imaging results, to handle the corresponding performance information, to express ideas and opinions, and to make decisions and solve related problems. [1, p. 124] We maintain that three primary conditions account for this lack of literacy. First, fifteen years of guidelines that have helped fuel ubiquitous digitization practices lack adequate attention to imaging science. Second, there is little or no community of practice in the cultural heritage sector today that focuses on the systematic use of existing and emerging image quality tools. Third, even for those users motivated to apply best practices, current standards and guidelines are sometimes difficult to locate, compare, and interpret. The technical literature is scattered, and information sometimes presents itself as a chaotic shopping list of unconnected links and at times cacophonous guidelines. There is also an abyss of technical knowledge between the small number of imaging professionals who have developed tools and specifications and the staff in libraries, archives, and museums who are expected to introduce these tools into their everyday workflows. Options for addressing the gap between scientific knowledge and professional performance have their strengths and weaknesses. On the job training through experimentation or apprenticeship may be limited by the availability of technical expertise in the nonprofit sector. Specialized training programs, such as those offered by IS&T and AIIM, may show immediate results in improved operations, but are limited in their reach and may be prohibitively expensive when tuition and travel costs are factored into the mix. A longer term strategy that could have significant and far-reaching impact is to infuse into the graduate school curricula of iSchools appropriate technical knowledge and a deep appreciation for the contribution of imaging science to the entire digitization quality package. iSchools are the training ground for cultural heritage professionals, so imaging industry partnerships can be viewed as a meaningful investment in future performance. This paper is a case study of one attempt to teach the rudiments of imaging science judgment in a course on cultural heritage digitization oriented toward future information professionals. The course is part of a curriculum on the preservation of information that itself is a component of a larger graduate degree program at the University of Michigan’s School of Information. The course utilizes a virtual education lab to provide access to Image Science Associates’ GoldenThread software, which is more typically used in imaging laboratories to assess the performance of scanning equipment. A practical, hands-on exercise, coupled with in-class lectures and discussion, reinforces skills in using the software while supporting students in their effort to map technical assessment to appropriate international standards and emerging digitization guidelines. Before describing the course and the imaging performance assignment, the paper contextualizes the course’s content in terms of preservation practices in the cultural heritage community and an emerging consensus on best imaging practices. The paper concludes with some preliminary findings on the outcome of the assignment and speculates on future work needed to improve the delivery of imaging science knowledge in an iSchool context. Digitization as a Preservation Strategy Expressing equal parts optimism about technological trends and anxiety about missing the wave of interest in digital reformatting, the Association of Research Libraries in 2004 endorsed digitization as a preservation reformatting strategy that could and should take its place alongside strategies for preserving “born digital” information. ARL acknowledged the incomplete development of standards governing digital preservation, but proclaimed that “libraries cannot wait for these solutions to be completely settled before testing the waters.” [2, p. 5] Then, in partial response to the spread of large-scale book digitization efforts, Erway & Schaffner engineered an impassioned call for cultural heritage professionals to “get into the flow,” in part by revisiting hard-won digitization practices developed for small scale “boutique” scanning projects. “Scaling up digitization of special collections will compel us to temper our historical emphasis on Archiving 2011 Final Program and Proceedings 65 quality... We must stop our slavish devotion to detail; the perfect has become the enemy of the possible.”[3, p. 6] Such expressions of policy and political expediency place a tremendous responsibility on the entire educational establishment to raise the bar on imaging science knowledge that is imparted to information professionals, who will be expected to cement the connections between digitization and preservation practices. Gracy and Croft see the educational value of engaging students in the dilemmas that the preservation field faces as it comes to terms with digital technologies. “As the library and archival professions grapple with preservation in the digital realm, education must keep students in the thick of the debate.” [4, p. 278] Yet the educational opportunities in the sphere of digitization are limited. In their survey of preservation education in iSchools carried out in 2005, Gracy and Croft found that 29 of 34 respondents lightly cover digitization in their general survey courses on preservation administration. Only five of the respondents offer a specialized course on digital preservation that may or may not include digital reformatting; none specifically on digitization practices. Three schools report having continuing education courses on digitization and digital libraries on the books, although the frequency of these offerings is not specified. Today, the University of Michigan may be one of the only graduate information programs to offer a dedicated course on digitization that focuses on preservation standards and best practices. The School of Information is an iSchool, such that faculty research and teaching interests meld themselves around the interdisciplinary concept of information. “The core vision [of iSchools] is that information, technology, and people are considered to interact and to be of roughly equal significance. Launching this required a decidedly interdisciplinary approach, with experts in each area sharing insights into meaningful syntheses of the three components.” [5, p. 17] Within the School, a group of faculty has coalesced around the idea that the preservation of information has a theoretical foundation, a nascent research agenda, and student demand that fuels curriculum development. [6] Imaging and Digital Literacy Imaging literacy enables good judgment, manifested as sound decision-making. In outlining the concept, Williams and Burns advocate for and document incremental progress toward economical and appropriate digitization decisions supported by imaging science whose very nature is technically sound, quantitatively based, and experimental. “Those responsible for the development of digital imaging content for libraries and museums will be well-served by acquiring a familiarity with the basic technology, characteristics and evaluation method of digital imaging.” [1, p. 127] Imaging literacy is made possible by emerging standards, national and international initiatives, and the development of testing and analysis techniques. At the level of imaging literacy in practice, Burns and Williams create a bridge between the science of imaging and the practice of digitization in the cultural heritage community by translating complex technical concepts into readily identifiable rules of thumb. [7] Stelmach and Williams expand upon this foundation by advocating for a statistical process control model in digitization that depends upon the savvy use of software for measuring the performance of scanning equipment over time and generating data for analysis. “By monitoring imaging performance with periodic tests (even image-by-image) one can identify potential trends early on, preventing not only rework but also embarrassment. ... The data is the foundation while the practices are the execution.” [8, p. 238] Imaging literacy is far more than a data-driven technical specification. As a theoretical construct, imaging literacy may best be allied with “digital visual literacy,” an emerging concept that attempts to reconcile a deep scholarly investigation of reading images [9] within the context of new digital media. [10] Lanham scoped out the concept near the beginning of the current era of digitization. “To be deeply literate in the digital world, means being skilled at deciphering complex images and sounds as well as syntactical subtleties of words,” he wrote. “Above all, it means being at home in a shifting mixture of words, images and sounds.” [11, p. 161] Bawden relates broad-based literacy concepts to the study of information. “It is not of importance whether this is called information literacy, digital literacy, or simply literacy for an information age,” he concludes. “What is important is that it be actively promoted as a central core of principles and practice of the information sciences.” [12, p. 251] Conway and Punzalan take a look at the role that digital literacy plays in the ways users create meaning from digitized photographs. [13] This article considers literacy issues from the educational perspective. The State of Imaging Guidelines The existence of appropriate guidelines and standards is a necessary precondition to building imaging literacy through classroom instruction. After fifteen years of systematic effort, it appears that the cultural heritage community now has access to imaging expertise and a variety of sophisticated guidelines for digitization that accomplishes preservation purposes. The first comprehensive documentation on appropriate practices emerged over ten years ago with the publication of Moving Theory into Practice. [14] Seven years later, Puglia and Rhodes reviewed the work to be done to address the connection between digitization processes and preservation practices and found a frustrating lack of progress. “It is a little humbling to look back and admit that we are still asking many of the difficult questions that we were asking over a decade ago – particularly about the relationship of digitization to preservation and agreement on approaches that are appropriate for preservation reformatting using digitization.” [15, p. 10] Conway investigated the development of digitization guidelines and found well-developed networks of expertise and the beginnings of convergence on comprehensive best practices. [16] The near simultaneous release of two important new guidelines documents is an expression of new levels of convergence and holds promise for influencing graduate level training. In August 2010, the 13-member Still Image Working Group of the Federal Agencies Digitization Guidelines Initiative (FADGI) released its Technical Guidelines for Digitizing Cultural Heritage Materials. [17] FADGI seeks to develop common digitization guidelines for historical and cultural materials that can be reproduced as still images, such as textual content, maps, photographic prints and negatives. The overall goals of the project are to enhance the exchange of imaging science research, encourage collaborative digitization practices and projects, and create publications of uniform quality. In addition to the new 66 ©2011 Society for Imaging Science and Technology guidelines document itself, the initiative has produced a nearcomprehensive international listing of digitization guidelines produced since 2000 and a catalog of appropriate digitization standards. In July 2010, the Koninklijke Bibliotheek (National Library of the Netherlands) released an advanced test version draft of guidelines for preservation imaging. [18] The guidelines provide tightly constructed technical criteria and tolerances for preservation imaging carried out in tandem with standardized scanner performance test targets. [19] Under the guidance of Hans van Dormolen, the guidelines have undergone a number of revisions and have recently been tested in with targets and software developed by Image Engineering (Germany). [20] The FADGI and Metamorfoze documents represent a triumph of imaging science over curatorial judgment in the digitization workflow, and yet the guidelines themselves are noticeably silent on the cumulative impact of many small technical decisions on the shape of the resulting digital product. In digitization workflow, quality standards and best practices play a crucial but relatively small role and quality assurance processes and procedures and are but one element. When coupled with assessment targets and software, the new guidelines provide an important tool in the digitization quality package. Stelmach and Williams highlight the value of imaging test targets and describe the underlying international standards that govern their design. [8] One weakness in the use of imaging performance targets and the accompanying software is support for interpreting the results of any given test. The current literature is notably weak in providing guidance when good scanning goes bad. These issues compound when imaging science enters the iSchool classroom. Lab Training in Imaging Practices A fundamental challenge in conveying science-based imaging literacy practices in an iSchool context – something shared by technologically oriented disciplines – is the provision to students of software resources to support image analysis. The challenge presents itself equally in classrooms, where technology laboratory space may be distant from the teaching classroom, inadequately equipped, or under-supported, and in distance education, where students need remote access to applications and support for collaborative learning. In both settings, licensing restrictions on use, technical limitations on simultaneous access, and weaknesses in technical documentation constrain the provision of software applications. The emergence of virtual technology laboratories is one possible solution to the two-pronged requirement to facilitate access to complex technology tools and support their use inside and outside the classroom. Although distance learning has been a driving force in the development of virtual computing labs, [21] colleges and universities have increasingly adopted a set of maturing technologies as a strategy for reducing the footprints of physical labs and providing direct access to students on their personal computers. [22] From a technological perspective, virtual labs may be centralized or decentralized. [23] In a decentralized model, students load specialized emulation tools on their computers, which then connect to an array of servers containing data. Decentralized virtual labs perform on student’s laptops; centralized labs perform through the network, either on a server or in the cloud. [25] If supplemented by thorough and appropriate documentation, virtual educational laboratories may serve as educational scaffolding that supports students as they learn to grasp theoretical principles and apply them to the practice of digitization in a laboratory environment. “Like training wheels, computer scaffolding enables learners to do more advanced activities and to engage in more advanced thinking and problem solving than they could do without such help.” [26, p. 214] At the University of Michigan, the School of Information relocated to new quarters in North Quad in 2010, prompting a review of the value of sprawling situated computer labs and the feasibility of leveraging a maturing suite of virtualization technologies in support of a sequence of technologically grounded courses. With funding from the National Endowment for the Humanities, faculty who teach a variety of courses in the area of archives and preservation are acquiring the technical support required to build, deploy, test, and perfect a robust virtual learning laboratory that is capable of supporting multiple courses in which more than 100 students are enrolled. University of Michigan Course on Digitization SI 675 Digitization for Preservation is a 1.5 credit hour course that meets three hours per week for seven weeks. It is one of ten courses offered by the School of Information (SI) in its Preservation of Information Specialization. [A] The learning objectives for SI 675 focus on understanding, interpreting, and applying standards and best practices for bitmap digitization; identifying emerging approaches to preservation quality imaging and digitization project management; and interpreting emerging descriptive and technical metadata standards for still images. The course includes two team-based assignments and a final examination. One assignment involves comparing and contrasting pairs of digitization guidelines developed since 2004. A second assignment engages students in imaging performance testing. Thirty-nine students completed the course in Winter 2011. The syllabus, reading list, and assignment outline is available through the University of Michigan’s open courseware initiative, Open Michigan. [B] P.A.V.E.L. for GoldenThread For the principal assignment, students gain remote access to Image Science Associates’ [C] GoldenThread software through the SI Preservation and Access Virtual Education Lab, P.A.V.E.L. [D] The approach to virtualization taken by SI is a centralized virtual lab powered by desktop virtualization. The virtualized desktop is hosted on a remote server that utilizes VMware vSphere Hypervisor (ESXi) virtualization tools to establish discrete operating system partitions for each course. Students authenticate to their course-specific server space in P.A.V.E.L. through a remote desktop client, after configuring a Virtual Private Network (VPN) on their laptop computers for added security. A VPN encapsulates data transfers using a secure cryptographic method between two or more networked devices that are not on the same private network. To access specific P.A.V.E.L. software, students download and save a small Remote Desktop Protocol (RDP) file (associated with Windows based server applications) for each application. The RDP file contains several parameters, including server address, port number, username, password, domain, desktop Archiving 2011 Final Program and Proceedings 67 size, and screen mode. The RDP file is compatible with Mac OS, Windows, and Linux operating systems. For the assignment, students used three RDP files, one each for the GoldenThread application, the database of device target results, and the database of object target results. Activating a given RDP establishes a secure connection to an individual application on the P.A.V.E.L. server. For additional security, students authenticate themselves with the P.A.V.E.L. server using established credential protocols at the University of Michigan. The project utilizes Microsoft’s Active Directory technologies, which are network service mechanisms that give server administrators a large degree of control over access permissions. The system supports the automated recognition of students enrolled in the specific courses that are part of the virtual lab project. GoldenThread Assignment The purpose of the imaging performance testing assignment is to reinforce foundational concepts of digitization quality, by utilizing the GoldenThread analysis software and associated device and object level targets. This assignment is partly a hands-on exercise using a flat-bed scanner or camera. The assignment is also an opportunity to synthesize the course readings and reflect on digitization quality metrics. Students work in pairs. Each team is responsible for carrying out the steps of the assignment and for developing jointly the deliverables. The assignment proceeded through six steps: 1. Identify a scanner or camera for testing from one of the many scanners available on the University of Michigan campus. 2. Digitize GoldenThread “Device-Level Target,” saving the results as either an uncompressed TIFF or standard JPEG format and recording information about the scanner and the software settings. The device target is a relatively large, standardized technical target used to qualify scanning equipment. 3. Digitize four diverse objects (e.g., photo, manuscript, map, book page) from the contents of a test packet on the same scanner used for imaging the test target, placing the “ObjectLevel Target” next to each item. The smaller object target, also standardized, is optimized for use with all items in a scanning batch. 4. Run the GoldenThread evaluation by connecting to the software located on P.A.V.E.L. Complete a full analysis on each of the saved files (1 Device and at least 4 Objects). 5. Compile data from the analysis in a spreadsheet. 6. Prepare and submit a report that describes the scanning equipment, interprets the results of the GoldenThread analysis, reflects on the quality assurance process, and comments on the exercise and the interpretation manual. An Interpretation Manual As with most commercially available software, GoldenThread is companied by a user manual that explains the functionality of the software and guides the user through configuration and operation, once the system has been loaded and configured on a single workstation. The manual is not adequate for an instructional setting because it contains little definitional information on the individual measures, does not provide context for applying the measures, and does not provide guidance on interpreting the results of the analysis. The default manual’s primary audience is staff in digitization labs who have responsibility for imaging quality assurance and who may be expected to have basic knowledge of scanner performance testing. For educational use, the authors of this paper revised the default manual to begin addressing the weaknesses of the default technical manual. The revised manual retains basic information on configuring the GoldenThread software for use, but significantly expands the explanatory content. A new section on “Creating Metrics and Interpreting Results” makes up over 50 percent of the revised manual. The section contains definitional information on tests for spatial frequency response, tone and color response, and noise and artifacts. The manual serves as a sort of scaffold for students as they learn to interpret results by comparing and contrasting the examples in the manual with the results from the virtualized software functioning on their personal desktop. The new manual illustrates normal output from an analysis and provides examples of what output looks like when the results suggest a departure from standards or best recommended practices. The manual then ties the information from these illustrations to the appropriate (if existent) technical literature Figure 1 us an example Figure 1. –Simple SFR graph with sharpening and imaging flare behaviors of how the manual highlights possible variation in the Spatial Frequency Response (SFR) curve. In this example, the manual annotates an idealized SFR curve and then ties the annotations to Williams and Burns’ explanation of the use of the Modulation Transfer Function. (25) The manual also selectively ties output results to FADGI recommendations. Figure 2 (below) is an example of how the manual annotates a standard GoldenThread output graph to tie scanner performance measures to the new US federal agencies digitization guidelines. Results During the Winter 2011 term, 17 teams of students conducted 22 tests on 14 discrete digital scanners located in eight separate locations on the University of Michigan campus. All but two of the scanning devices were publically accessible, the others being located in restricted access scanning laboratories. Various models of Epson flatbed scanners, controlled by Epson Scan software, dominated the evaluation exercises. Only one relatively high-end scanning device (CopiBook HD 400) and one digital SLR (Canon 68 ©2011 Society for Imaging Science and Technology D-50) were utilized in the test. Students typically conducted the full suite of test procedures in GoldenThread, using the ISa Device-Level Target. Because the students did not conduct tests in an optimized environment, the exercise could not fully mimic laboratory quality control processes. Figure 2 –FADGI guidelines with respect to several SFR evaluation points The resulting reports from the student teams vary widely in quality. At the low end of quality, student reports sometimes serve mostly to demonstrate that the scanning exercise has been completed successfully and that the output from the experience could be mapped to the appropriate parts of the manual. These reports treat the manual as a textbook instead of a heuristic device. The most successful assignment reports from the GoldenThread analysis have three distinctive features in common. First, team members reach well beyond the information in the manual to include analysis and interpretation from the secondary imaging science literature – almost all of which was listed in the course syllabus or in two supplemental guides. Second, team members grasp the limitations of the data provided in the analysis output. The results of some of the GoldenThread analytics could be mapped successfully to the performance of the individual scanner, whereas the results of other tests were not valid without further testing. Third, the best team reports include sophisticated reflection on the role that scanner performance testing plays in the overall quality assurance package. The several highly successful team reports juxtapose data analysis from GoldenThread with visual inspection of scanned images and articulate the limitations of an analysis that derives from incomplete knowledge of scanner technologies, monitor calibration, and other environmental factors not well controlled in a classroom exercise. As for the manual itself, it remains a work in progress, fundamentally limited by the gaps in the research literature documenting variations on the outcomes of performance testing. A technical manual or textbook can only be as good as the foundational research base and the expertise of the authors. There remains no question in the minds of the authors of the revised GoldenThread manual that the significant gaps in the literature on interpreting the results of performance testing is a barrier to effective classroom teaching. The P.A.V.E.L. delivery environment for GoldenThread functioned consistently as designed. The entire project benefitted from dedicated programming services of a systems administrator skilled in the use of virtualization tools. Students nevertheless experienced a variety of difficulties that can be traced to three sources. The first limitation of P.A.V.E.L. is that GoldenThread is not designed to function in a multi-user environment. GoldenThread requires some time to load very large image files and students usually lack patience to wait for a slow feedback loop. Second, log-in and log-out procedures did not permit multiple simultaneous uses of the software, which taxed student groups used to completing assignments at the last minute. The third limitation of P.A.V.E.L. is the mental model that it conveys to students who have limited experience with non-routine desktop applications. To use the virtual laboratory most effectively, students need to create a mental schematic of the ways that data move from desktop to server to software and back to the desktop. Such a schematic could be scaffolded with system diagrams. Beyond the provision of imaging science software tools and interpretive resources, the course dealt with pedagogical challenges in teaching imaging science concepts to graduate students who will not themselves become imaging scientists. In the classroom it became clear that imaging science itself provides a viable framework for bridging two points; the first being the goals of preservation quality digitization, the second being the technical mechanics of the digitization process. Additionally, we found that the fixed structure of the GoldenThread software that features embedded processes, linear processing routines, and a general intolerance for error did not always mesh well with the dynamic nature of the classroom lecture/discussion experience. The concrete virtual laboratory experience helped students find the right balance between technical detail (depth) and the socio-political context (breadth) of cultural heritage digitization. A combination of experience and inclinations that students bring into the classroom influences this balance. Students varied widely in the level of technical expertise, ranging from sophisticated (programming, database development, digitization) to novice. Our perspective on imaging literacy demands that cultural heritage professionals obtain a foundation of imaging science concepts, yet students differ in their propensity and willingness to absorb technical information. The relationship between pedagogical goals and student aptitudes is a matter that we are studying further as part of our ongoing virtual laboratory experiments. Conclusion In their review of trends in virtual labs, Gaspar et al. claim that “remote accessibility of the students’ virtual machines does not impact learning directly but [merely] improves flexibility in the usage of campus resources.” [22, p. 127] The experience of using GoldenThread software in a virtual laboratory environment suggests that learning may actually intensify when classroom discussion and demonstration are tied explicitly to self-directed learning with the use of technology tools. The point of the exercise was not to teach or promote GoldenThread, but rather to demystify scanner performance testing and provide an example of how digital imaging standards and best practices play out in practice. The Archiving 2011 Final Program and Proceedings 69 results of the assignment suggest that students grasped these points, even if they did not always master the intricacies of the technology itself. Delivering GoldenThread dynamically through a virtual lab provided evidence of a scaffolding effect for some students, beyond the obvious benefits of 24*7 remote access. In striving to impart “imaging literacy” to graduate students who are not now and do not aspire to become imaging scientists, educational design is a very important success factor. Students must understand imaging technologies by using the tools that are emerging to support digitization practice. Educators working with imaging scientists must decide which elements of a complex scientific discipline are appropriate for coursework oriented toward future cultural heritage professionals. Together, we must also establish the best ways to impart these elements, while retaining the organizational and administrative context within which imaging science concepts will be applied. Most important, imaging science education in iSchools must avoid Alexander Pope’s admonition from three hundred years ago that critical judgment can only come from the sobering effects of deep learning. [27] For at the end of the day, digitization practice that addresses preservation issues is really good judgment disguised as technologically mediated decision-making. Acknowledgements The US National Endowment for the Humanities (PE-5004310) supported the project. The authors wish to thank co-PIs David Wallace and Elizabeth Yakel, Franz Agaz at the University of Michigan’s School of Information Computer Services and Steve Tomanovich at Image Science Associates for technical support.